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A rigorous definition is given for the concept of an "interaction matrix' 1 (Zy) where i = 1 to m 
and /— 1 to n, in terms of two idernpotent matrices A, and B s of rank r and 5, respectively. It is then 
shown that the frequency distribution of the eigenvalues of {Z)(Z)' depends only on r and s. Applica- 
tions are given to matrices of residuals arising from two-way data, either by removing row and/or 
column-means, or by applying any number of sweeps of the "vacuum cleaner." The theorems are 
important in the theory of the analysis of two-way tables of nonadditive data. 
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1. Introduction 

In a recent paper [l], 1 a method has been presented for a partitioning of the row by column 
interaction in two-way tables and the practical usefulness of this method as a tool in data analysis 
has been demonstrated. The method required the calculation, by Monte Carlo techniques, of 
quantities that are analogous to the "degrees of freedom" in ordinary analysis of variance. These 
new degrees of freedom were calculated as follows. 

Let (Zij) be an m X n matrix of independent random normal deviates (zero mean, unit variance). 
Consider the m X n matrix of residuals (dij), where 

("ij £* ij i *•* . . ** i . £* . j ? 

where a dot indicates averaging over the index replaced by the dot. 

Then the new quantities, denoted as degrees of freedom, are the expected values of the 
eigenvalues of the matrix product (d) (d)' (where ' indicates the transpose of a matrix). The 
expected values of these eigenvalues are functions of m and n, and were tabulated as such. Pro- 
fessor John Tukey, of Princeton University, has pointed out that the degrees of freedom calculated 
by this procedure are of far more general applicability than indicated in the paper in question. 
The present paper provides the theoretical development from which the more general results are 
obtained. It is based on a new, mathematically rigorous definition for an interaction matrix, from 
which an important general result can be derived. 

2. A Definition of Interaction of Two-Way Tables 

Let (djj) be an m X n matrix of normally distributed variates, all of zero mean; and let the 
covariance of d^ and dry be denoted by cov(d,j, dry), 

We will say that (dij) in an interaction matrix of r degrees of freedom by s degrees of freedom 
if cov (djj, di'y) is of the form 

co\ (dij, di>f) = [a-iv ■ bjf] o* 2 , 



1 Figures in brackets indicate the literature references at the end of this paper. 
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where the mXm matrix A r = (aw) is idempotent of rank r, and the nXn matrix B s =(bjj>) is 
idempotent of rank s. 

We will say that A r and B s are the covariance matrices associated with the interaction matrix 

(dtj). 

3. The Eigenvalues of (d)(d)' 

THEOREM I: //(dy) is an mX n interaction matrix with associated covariance matrices A r and 
B s , then the nonzero eigenvalues of the matrix product (d)(d)' (where (d)' denotes the transpose 
of (d)) are the same as those of the matrix (t)(t)', where (ty) is an rX s matrix of normally and inde- 
pendently distributed variates t ij9 of zero mean and common variance or 2 . 

PROOF: LetP = (pu>) represent the m X m. orthogonal matrix whose rows are the eigenvectors 
of A r . Since A r is idempotent and of rank r, we have 



PAJ "-\o 



1,0 



or 



(2) 



where I r is the identity matrix or rank r. Similarly, representing by Q = (qjy) the n X n orthogonal 
matrix whose rows are the eigenvectors of B s , we have 

From (2) and (3) we derive, respectively, 

^> x-i -, f 8w for i, i ^ r 

?? W,8 ""l0 foriori'>r ,4) 
and 

v^ v-« i f Sjj' for 7,/ ^ 5 

T T 10 for 7 ory > 5, w 

where S^ is the Kronecker delta (8 ki = 1 for k = /, and 8a-; = for A: 7^ /). 
Consider now the transformation 

{tf>=P{d ij )Q'. (6) 

Then, 

(t*)(t*y=p(d)Q'Q(dyp' 

or 

(t*)(t*y=P(d)(d)'P'. (7) 

From (7) it follows that (t*) it*)' and (d) (d)' have the same nonzero eigenvalues. Furthermore, we 
have, on account of (6) 

m n 

^SJPttWfc-. (8) 

Consequently, * u 



m n m n 



CO\(t%, tf f ) = 2 2 2 2 PikQjvPi'Klj'v COV (rf A . M , rf^). (9) 



Introducing (1) into (9), we obtain 



m n m n 



cov (ig , $, ) = <x 2 ]£ ^] 2 2 PikQjuPi'iqrva k! b uv 

k u I v 

= °" 2 ( 2 2 pkpv&m ) ( 2 2 qjuqj'v°™ ) 
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which, 


as 


a 


result of 


eqs (4) and (5), 


becomes: 






















cov (*g, tf, f ) 


= 


(T 2 




•8«< 


•o.// 


for 
for 


li 


/ 

or 

or 


I 
f 


r 
s 

> r 
>s. 



(10) 



From (8) it also follows that the ty are normally distributed with E(t*)=0 for all i and j. Con- 
sequently, because of (10), the ty are equal to zero for all i, i > r or j, / > s. For /, i' ^ r and 
j, / ^ s, it follows from (10) that the t^ are normally and independently distributed with zero 
mean and variance cr 2 . 

We have already shown that (d)(d)' and (t*)(t*)' have the same nonzero eigenvalues. If 
we omit from the matrix (t*) , all elements for which i > r, ory > s (all of which are zero), we obtain 
an rXs matrix, say (£y), such that (t)(t)' has the same nonzero eigenvalues as (**)(**)', and 
consequently as (d) (d)'. QED. 

NOTE: We have assumed, in the definition of an interaction matrix, that the dtj are normally 
distributed, and have proved that under this assumption, the tfj defined by eq (8) (except for those 
that are identically zero), are independently and normally distributed with zero mean and variance 
(j 2 . We can generalize the definition of an interaction matrix by requiring only that the a\j be con- 
ditionally normal, given the an* and bjy. Since the distribution of the ty does not depend on the 
aa> and by, it is unconditionally normal with zero mean and variance cr 2 , and Theorem I remains 
true under the more general definition of an interaction matrix. 

4. Residuals from an Additive Structure 

THEOREM li: Let (dy) be a matrix whose elements are 

i . . ~ fi=l torn 

dij-yu-M-Pi-tt l J = lton («) 

where y u are independent normal variates, such that E(yij) = fi + Pi + yj, and whose common variance 
is cr 2 , and where /i, pi, % are the usual estimates of the grand mean? the row effects and the column 
effects. Then (d ) is an interaction matrix of(m — 1) degrees of freedom by (n — 1) degrees of freedom. 
PROOF: It is readily shown from (11) that 

cov (d ih */) = (fe~)(%r-J)o«. (12) 

The matrix A = ( Sn* lis idempotent. Its trace is ^ ( da J = m — 1 . Hence its rank is m — 1 . 

Similarly, the matrix B= ( S> j is idempotent, of rank n—\. This proves the theorem. 

5. Residuals from the Vacuum Cleaner 2 

Lemma I: Let (y^) be an m . X n matrix of observations Vy, wijA common variance cr 2 . Le£ (dij, k ) 6e 
j/ie matrix of residuals obtained from the y u «/£er extraction of the grand mean (/x), ^e row <#ec£s 
(Pi), the column effects (yj), and k sweeps of the vacuum cleaner [3]. 77ierc the following relation 

holds: 

k 

dij,k = Yij - (£ + Pi + 7i) " 2 [^iPit^U + "li^U + ^ijP"] ( 13 ) 



2 The results contained in Lemmas I, II, and III are not novel. They have been derived previously, by a different method, by W. L. Nicholson [2], and are pre- 
sented here only for completeness. 
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in which 





2>» 


= S * = 


= 


(1)1 


1 


J 






i 


j 


= 1 


(2) 


?n= 


Pi 





for all /. 



qij=- 



VSp* V^rf 



(3) Pu = 



gi-Li 



2<i.i 



qu 



_ j3i-i,j 



S«-i.j 



(4) 



i j 

«U = 2 d iJ 5 l-l ( IlJ _K lPli 
j 



These relations result from the definition of the vacuum-cleaner process (see [3]), according to 
which 

dij, i = dtjj-x — [Kipuqtj + anqij + /%>«] , (14) 

and the quantities K, p, q, a, ft, are calculated as above. 
Lemma II: Using the notation of Lemma I, we have 



and 



2 PliPl'i = fill' 
i 

2 qijqi'j = 8u< 



/or all 1, 1' 



(15) 



(16) 



PROOF: Let (t) = (t u t 2 \ . . ., t n ) be any vector of n elements, such that V = 0, and assume 
that the following relations hold for some given value of /. J 



(1)J*0» + U = O 

j 

(2) ^ #«,,, = 



for all i, 



where dijj satisfies eq. (14) of Lemma I. 
Then the following relations hold: 



(a)2*j«fi+t.j = 

j 

(b)]£ *A,*+r=0 for all i. 
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Proof: 



(a)2^ + 2,j = 2o 



ft+u 



1 



i 



2 2 (l ij,lPl+\,i — Ki+iQi 



+ lj 



X^ /+1 '' (2 *Mm) _ ^+l 2 0^«+l J 



0, because of (1) and (2). 



(6) 2 tjdij,i + i = ]? tjdij,i — Ki +u pi +1 ,i ^ tjqi+i,j 

J J j 

-%i,i^] tjQi+uj — Pj+m 2 tjfr+ij = 0, because of (1), (2), and (a). 
j j 

Thus, if conditions (1) and (2) are fulfilled for any value of /, they are also fulfilled for any value 
/' > /. In that case, we have 

J^/'i^O forZ'^Z+1. 

If we now make fy = qij, we can verify that conditions (1) and (2) are fulfilled. Hence, 

2wiy=0 for/'*/, 

j 

and by definition V QvQtj— 1- The relation ^pnpn = $ir is proved in a similar way. 
j ' 

Lemma HI: Defining (dy, k ) as £/i Lemma /, we /m#e 

coi; (d ijik ,d iT , k )=o- 2 f 8 ir - — - JPuPu'J {^y~ ~~ S q « q «')' 

PROOF: The lemma is easily verified for k= 1. It is also readily shown, using Lemma II, that 
if the lemma holds for k, it also holds for A + 1. Thus, the lemma is proved by mathematical induc- 
tion on k. 

THEOREM III: Define (djj, k ) as in Lemma I, and assume furthermore that the yy are normally 
and independently distributed, with E(y ij ) = ^t + p i H-yj, and common variance a 2 ; then the m Xn 
matrix (dy, k ) is an interaction matrix of (m — k — 1) by (n — k — 1) degree of freedom. 

PROOF: The theorem results at once from Lemma III amd the fact that a matrix of the form 



with 






^PuPvi — 8a> 



is idempotent and of rank m — k — 1. (The trace of this matrix is 



&i- 



1 



Spfi 



lm' 



;! The distribution of the djj, k is only conditionally normal, given all pn and qij values, but as shown on p. 151, this does not invalidate Theorem I. Consequently, 
Theorem III is true. 
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THEOREM IV: Let (y u ) be an mX n matrix of normal, independent variates yy, whose means are 
E(yij)— M^Pi + Tj an d whose common variance is a 2 . Extract from y u the mean pi, the row effect 
p i5 and the column effect y- } , as well as k sweeps of the vacuum cleaner, and denote by (d ijjk ) the 
matrix of residuals resulting from this treatment. 

Then the nonzero eigenvalues of the matrix product (dij t k)(dij f k)' are the same as those of 
(0(0' where (0 is an (m~k~l) by (n — k—1) matrix (ty) of normal, independent variates ty of 
zero mean and common variance cr 2 . 

The theorem holds for all values of k, from k = to k= min [ (m — I ) , (n— 1 ) ] . 

PROOF: The theorem is an immediate consequence of Theorems I and III. 

6. Principal Result and Summary 

As indicated in the introduction, the present work was motivated by a desire to generalize 
results obtained in connection with a new method of data analysis. We are now in a position to 
state this generalization in the following precise form. 

Theorem V: Let (dy) be an mX n interaction matrix of r degrees of freedom by s degrees of 
freedom. Then the probability distribution of the eigenvalues of the matrix product (d)(d)' depends 
only on r and s, and not on m and n. 

PROOF: The theorem follows at once from Theorem I. 

Our principal result is the following: To study the probability distribution of the eigenvalues 
of (d)(d)[ where (d) is any interaction matrix with r by 5 degrees of freedom, it is sufficient to 
study the probability distribution of the eigenvalues of (t) (t) ', where (t) is an r by s matrix whose 
elements are independent random normal deviates (mean zero, unit variance). The probability 
distribution in question is completely determined by the values of r and s. 



The author is greatly indebted to Professor John Tukey for pointing out to him the generality 
of his previous results and thereby providing the incentive for the present work. 
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